AD-A114  478  WISCONSIN  UNIV-MAOISON  MATHEMATICS  RESEARCH  CENTER  F/6  6/5 

ASSESSING  SENSITIVITY  TO  AN  UNOBSERVED  BINARY  COVARIATE  IN  A  NO— ETC (U) 
JAN  82  P  R  ROSENBAUM*  0  B  RUBIN  DAA629-a0-C-0041 

UNCLASSIFIED  MRC-TSR-2321  NL 


woo  m  M 


HRC  Technical  Sunnary  Report  #  2321 


ASSESSING  SENSITIVITY 
TO  AN  UNOBSERVED  BINARY  COVARIATE 
IN  A  NONRANDCMIZED  EXPERIMENT 
WITH  BINARY  OUTCOME 


Paul  R.  Rosenbaum 
and 

Donald  B.  Rubin 


Mathematics  Research  Center 
University  of  Wisconsin— Madison 
610  Walnut  Street 
Madison,  Wisconsin  53706 

January  1982 

(Received  November  9,  1981) 


Appravsd  for  public  roloaso 
Distribution  unlimited 


DTIC 

(EUECTE 
MAY  181982 


Sponsored  by 

U.  S.  Army  Research  Office 
P.  O.  Box  12211 
Research  Triamgle  Pairk 
North  Carolina  27709 


National  Cancer  Institute 
9000  Rockville  Pike 
Bethesda,  MD  20205 


Health  Resources 
Administration 
3700  East-west  Highway 
Hyattsville,  MD  20782 


ACKNOWLEDGEMENT 


The  authors  wish  to  acknowledge  valuable  discussions  with  Arthur  P. 
Dempster  emd  Roderick  J.  A.  Little  on  the  subject  of  this  paper. 


UNIVERSITY  OF  WISCONSIN  -  MADISON 
MATHEMATICS  RESEARCH  CENTER 


ASSESSING  SENSITIVITY  TO  AN  UNOBSERVED  BINARY  COVARIATE 
IN  A  NONRANDOMIZED  EXPERIMENT  WITH  BINARY  OUTCOME 

Paul  R.  Rosenbaum*  and  Donald  B.  Rubin** 

Technical  Summary  Report  #2321 
January  1982 
ABSTRACT 

■J 

The  results  of  nonrandomized  clinical  experiments  are  often  disputed 
because  patients  at  greater  risk  may  be  overrepresented  in  some  treatment 
groups.  This  paper  proposes  a  simple  technique  providing  insight  into  the 
range  of  plausible  conclusions  from  a  nonrandomized  experiment  with  binary 
outcome  and  observed  categorical  covariate.  The  technique  assesses  the 
sensitivity  of  conclusions  to  assumptions  about  an  unobserved  binary  covariate 
relevant  to  treatment  assignment,  and  is  illustrated  in  a  medical  study  of 
coronary  artery  disease.^ 
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ASSESSING  SENSITIVITY  TO  AN  UNOBSERVED  BINARY  COVARIATE 
IN  A  N0NRAND0MI2ED  EXPERIMENT  WITH  BINARY  OUTCOME 


*  ** 

Paul  R.  Rosenbaum  and  Donald  B.  Rubin 

1.  INTRODUCTION  AND  NOTATION 

Inevitably,  the  results  of  clinical  experiments  are  subject  to  dispute. 

In  nonrandomized  experiments,  one  basis  for  dispute  is  obvious:  since 
patients  were  not  assigned  to  treatments  at  random,  patients  at  greater  risk 
may  be  overrepresented  in  some  treatment  groups.  This  paper  proposes  a  method 
for  assessing  the  sensitivity  of  conclusions  to  an  unmeasured  patient 
characteristic  relevent  to  treatment  assignment.  Despite  their  limitations, 
nonrandomized  experiments  will  continue  to  be  performed,  and  therefore  it  is 
prudent  to  develop  appropriate  methods  of  analysis  for  them. 

Our  sensitivity  analysis  consists  of  the  estimation  of  the  average  effect 
of  a  treatment  on  a  binary  outcome  variable  after  adjustment  for  observed 
categorical  covariates  and  an  unobserved  binary  covariate  u,  under  several 
sets  of  assumptions  about  u.  Both  Cornfield,  et.  al  (1959)  and  Bross  (1966) 
have  proposed  guidelines  for  determining  whether  an  unmeasured  binary 
covariate  having  specified  properties  could  explain  all  of  the  apparent  effect 
of  a  treatment;  i.e.,  whether  the  treatment  effect,  after  adjustment  for  u, 
could  be  zero.  Our  method  has  two  advantages:  first.  Cornfield,  et.  al. 
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(1959)  and  Bross  (1966)  adjust  only  for  ths  unneasured  covariate  u,  whereas 
we  adjust  for  measured  covarlates  in  addition  to  the  unmeasured  covariate 
u}  second.  Cornfield,  et.  al.  (1959)  and  Bross  (1966)  only  judge  whether  the 
effect  of  the  treatment  could  be  sero  having  adjusted  for  u,  whereas  we 
provide  an  actual  estimate  of  the  treatment  effect  adjusted  for  u  as  well  as 
the  observed  categorical  covariates. 

In  principle,  the  i^  of  the  N  patients  under  study  has  both  a  binary 
response  r^^^  that  would  have  resulted  if  he  h2ul  received  the  new  treatment, 
and  a  binary  response  r^^  that  would  have  resulted  if  he  h^u3  received  the 
control  treatment.  In  this  formulation,  treatment  effects  are  comparisons 
of  r^^  and  rQ^,  such  as  Since  each  patient  receives  cmly  one 

treatment,  either  r^^  or  rQ^^  is  observed,  but  not  both.  Therefore, 
comparisons  of  r.^^  and  rQ^  imply  soma  degree  of  speculation.  Treatment 
effects  defined  as  comparisons  of  the  two  potential  responses,  r^^  and 
rgj^,  of  individual  patients  are  implicit  in  Fisher's  (1953)  randomization 
test  of  the  sharp  null  hypothesis  that  i  **  Such 

definitions  are  used  explicitly  by  Kempthorne  (1952)  in  his  discussion  of 
randomization  based  inference,  and  by  Rubin  (1977,  1978),  Hamilton  (1979),  and 
Rosenbaum  and  Rubin  (1981)  in  discussions  of  observational  studies.  (The 
definition  does  contain  boom  implicit  assumptions,  such  as  the  assumption  of 
noninterference  between  patients;  see  Cox  (1958,  chapter  2)  or  Rubin  (1978, 
section  2.3)  for  discussion.)  Here,  the  N  patients  in  the  study  are  viewed 
as  a  simple  random  sample  from  some  population,  and  the  average  treatment 
effect  is  defined  as 

B(r^)  -  E(rg)  -  prob(r.|  -  1)  -  prob(rg  -  1)  ■  -  T^,  say  (1.1) 

where  E(«)  and  prob(*)  denote  expectation  and  probability,  respectively, 
in  the  population. 
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For  the  patient  of  N  patients  in  the  study  (i  ■  let 

be  the  indicator  for  treatment  assignment,  and  let  <■  1  if  patient 
i  is  assigned  to  the  new  treatment,  and  z^  >  0  if  patient  i  is  assigned 
to  the  control  treatment.  Suppose  that  patients  have  been  stratified  or 
subclassified  into  one  of  J  strata  on  the  basis  of  observed  covariates,  and 
that  patient  i  falls  in  strattim  s^.  The  population  model  for  the 
stratifying  variable  s  is  assumed  to  be  a  saturated  multinomial. 

In  a  stratified  randomized  trial,  treatment  assignment  z^  and 
response  rQ^}  are  known  to  be  conditionally  independent  given  the 

stratum  s^,  or  in  Dawld's  (1979)  notation: 

<>=■11'  *^0i’  -Li-  ^i  •  ®i  ' 

l.e.,  within  strata,  treatment  assignment  is  "random".  Condition  (1.2)  is  not 
kno%m  to  hold  in  a  nonrandomized  experiment.  Generally,  we  shall  say 
treatment  assignment  is  strongly  ignorable  given  some  set  of  covariates  _v  if 

<ru»  'oi>  -iL*i  I  Xi  • 

For  brevity,  when  treatment  assignment  is  strongly  ignorable  given  the 
observed  covariates  (i.e.,  when  as  in  a  randomized  experiment,  (1.2)  holds), 
we  shall  say  simply  that  treatment  assignment  is  strongly  ignorable. 

(Technical  Note:  If  treatment  assignment  is  strongly  ignorable,  then  it  is 
ignorable  in  Rubin's  (1978)  sense,  which  only  requires  that  the  probabilities 
be  evaluated  at  observed  outcomes;  however,  the  reverse  is  not  true  since 
strongly  ignorable  Implies  the  relationship  among  probabilities  must  hold  for 
all  possible  values  of  the  random  variables.) 

We  develop  and  apply  a  method  to  aid  in  judging  the  sensitivity  of 
conclusions  to  certain  plausible  variations  in  assumptions  about  an  unobserved 
binary  covariate  u.  In  particular,  we  assume  that  treatment  assignment  is 
not  strongly  Ignorable  given  s,  but  is  strongly  Ignorable  given  s  and 


ui  l.e.,  that  (1.2)  is  false,  but  Instead  that 

I  ®i'  (1»3) 

is  true.  If  conclusions  are  insensitive  over  a  range  of  plausible  assumptions 
about  u,  the  number  of  interpretations  of  the  data  is  reduced,  and  causal 
conclusions  are  more  defensible. 
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2.  AN  EXAMPLE  USING  DATA  ON  CORONARY  ARTERY  DISEASE 


Before  presenting  the  model  underlying  the  sensitivity  analysis,  we  show 

the  results  of  applying  it  to  an  example  concerning  symptomatic  relief  from 

* 

coronary  artery  disease.  Treatment  1  is  coronary  artery  bypass  surgery; 
treatment  2  is  medical  therapy.  The  response  (r^,  rg)  is  functional 
improvement  six  months  after  cardiac  catheterization,  with  r^  =  1  signifying 
improvement  under  treatment  t,  t  =  0,  1 ,  and  r^.  =  0  signifying  no 
improvement. 

Here,  patients  are  stratified  by  the  multivariate  method  described  in 
Rosenbaum  and  Rubin  (1981)  that  balances  74  observed  covariates.  The  observed 
proportion  improved  within  each  stratum  is  displayed  in  Table  1.  In  examining 
this  table,  it  must  be  remembered  that  there  exists  evidence  for  a  placebo 
effect  of  bypass  surgery  (Benson  and  McCalie  (1979)). 

When  treatment  assignment  is  strongly  ignorable  given  the  stratum  s, 
then  direct  adjustment  with  stratum  total  weights  yields  the  maximum 
likelihood  estimate  of  the  average  treatment  effect  (1.1)  under  the  saturated 
multinomial  model  for  s.  The  directly  adjusted  proportions  improved  are  .36 
for  medicine  and  .67  for  surgery,  with  standard  errors  .04  and  .05 
respectively  (calculated  following  Hosteller  and  Tukey,  1977,  ch.  11c). 

In  order  to  study  the  sensitivity  of  estimates  to  the  assumption  of 
strongly  ignorable  assignment  (i.e.  to  assumption  (1.2)),  we  now  assume  that 
treatment  assignment  is  not  strongly  ignorable,  but  rather  that  treatment 
assignment  is  strongly  ignorable  given  s  and  u  (i.e.  that  assumption  (1.3) 
holds),  where  u  is  an  unobserved  binary  covariate.  Table  2  displays  the 
sensitivity  of  the  estimate  of  the  average  treatment  effect  to  12  sets  of 
assumptions  about  u.  The  method  used  to  derive  Table  2,  and  the  symbols  that 
appear  in  Table  2,  will  be  mathematically  defined  in  the  next  section. 


* 
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Table  1 


Proportion  of  Patients  Improved  at  6  Months  In  Each  Stratum 


Stratum* 

Treatment 

Humber  of 

Patients 

Proportion 

Improved 

Standard 

Error 

1 

Medical 

277 

.35 

.03 

Surgical 

26 

.54 

.10 

2 

Medical 

235 

.40 

.03 

Surgical 

68 

.70 

.06 

3 

Medical 

205 

.35 

.03 

Surgical 

98 

.70 

.05 

4 

Medical 

139 

.30 

.04 

Surgical 

164 

.71 

.04 

5 

Medical 

69 

.39 

.06 

Surgical 

234 

.70 

.03 

* 

The  strata  were  constructed  by  the  aethod  of  Rosenbaum  and  Rubin  (1981)  in 
which  the  conditional  probability  of  surgical  treatment  given  the  observed 
covariates  is  estimated,  and  patients  with  similar  estimated  probabilities  are 
placed  in  the  same  stratum.  The  strata  are  predictive  of  treatment 
assignments!  they  are  not  prognostic  strata*  This  method  balances  observed 
covarlates  within  each  stratum. 


Roughly  speaking,  a  Is  the  Increase  in  the  log  odds  of  surgery  associated 


with  u  =  1  rather  than  u  =  0;  6^  is  the  increase  in  the  log  odds  of 
improvement  under  treatment  t  associated  with  u  =  1;  and  ir  defines  the 
t  marginal  distribution  of  u  by  ir  -  p(u-O). 

In  Table  2,  the  estimates  of  the  proportion  improved  vary  from  .34  to  .38 
for  medicine  and  from  .63  to  .70  for  surgery;  this  range  of  values  is  about 
the  same  as  the  standard  error  of  the  directly  adjusted  proportions  (i.e.,  .04 
for  medicine,  .06  for  surgery).  Consequently,  we  see  that  this  hypothetical, 
unobserved  covariate  u,  which  has  defied  the  cardiologists'  attempt  to 
record  all  variables  used  in  assigning  treatments,  would  have  to  more  than 
triple  the  odds  of  surgery  and  more  than  triple  the  odds  of  improvement, 
before  altering  the  conclusion  that  the  proportion  Improved  under  surgery  far 
exceeds  the  proportion  improved  under  medicine.  Although  this  difference  may 
reflect  a  placebo  effect  of  surgery,  the  difference  does  not  seem  to  be  easily 
explained  as  the  result  of  an  imbalance  due  to  the  nonrandomized  nature  of  the 
study. 

Another  way  of  describing  this  analysis  is  to  say  that  we  have  explored 
the  extent  to  which  the  data  might  be  an  example  of  Simpson's  paradox.  For 
discussion  from  this  perspective,  see  Lindley  and  Novick  (1981). 
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3.  THE  MODEL  AND  THE  MAXIMUM  LIKELIHOOD  ESTIMATE 


This  section  describes  the  model  underlying  the  sensitivity  analysis  in 
Table  2;  section  4  describes  computational  procedures.  The  principal  result 
of  this  section  is  the  derivation  of  the  maximum  likelihood  estimate  of  the 
average  treatment  effect  (1.1). 

Assume  treatment  assignment  is  strongly  ignorable  given  s  and  u 
(i.e.,  assume  (1.2)).  Let  ir  define  the  marginal  distribution  of  the 
unobserved  covariate  u  by 


p(u=0)  =  IT 
p(u=1 )  =  1  -  It 

where  for  convenience  u  is  assumed  independent  of  s;  i.e., 

u  I  I  s  .  (3.1  ) 

As  stated  previously,  the  population  distribution  of  the  stratifying  variable 
is  assumed  to  be  a  saturated  multinomial  for  the  J  possible  values  of  s. 
Assume,  for  treatment  assignment 

exp{z(Yg  +  otu)} 

p(z|s,u)  °  exp{Y^  ";  cu}'  ' 

and  assume  for  response  under  treatment  t,  t  =  1,2 

exp{r  (3  +  6  u)} 

p(^^i|s,u)  -~~i7prg;~t'j';u}  * 

where  Y  ,  3  . ,  s  =  1,...,J,  t  ■  0,1,  are  parameters  to  be  estimated,  and 
s  St 

Cl,  ,  TT  are  parameters  assigned  values  for  the  sensitivity  analysis. 


In  words,  the  parameter  a  defines  the  relationship  between  the 

unobserved  covariate  u  and  treatment  assignment  zt  the  parameter 

defines  the  relationship  between  u  and  the  response  r^  to  treatment  t, 

t  -  0,1.  within  every  stratum,  u  -  1  Increases  the  log  odds  of  receiving 

treatment  1  by  the  same  amount  a.  Similarly,  within  every  stratum, 

u  «  1  increases  the  log  odds  of  io^rovement  under  treatment  t  (i.e.,  of  r^ 

-  1 )  by  the  same  amount  6^.  (me  development  below  can  also  be  immediately 

applied  to  the  case  where  s,  and  a  are  replaced  by  stratum  dependent 

parameters  *  ,  ^  and  a  ,  %ihlch,  in  particular,  do  not  satisfy 
8  St  s 

assumption  (3.1);  however,  this  more  general  formulation  introduces  too  many 
sensitivity  parameters  for  many  practical  applications. ) 

For  fixed  values  of  the  sensitivity  parameters,  a,  v,  the 

maximum  likelihood  estimates  (mle*s)  of  the  3J  parameters  Y  ,  B  0  ,,  s  « 

S  so  SI 

1,2,...,J,  do  not  have  a  simple,  closed  form.  However,  there  are  3J 
parameters  that  are  a  1-1  function  of  Y  ,  0  3  .  that  do  have  simple, 

S  so  SI 

explicit  mle's,  namely,  p(s«>o|8),  p(rQ»o|z«0,s)  and  p(r^*0|z>1 ,s).  We 

first  derive  the  relationship  between  these  two  sets  of  parameters,  then 

display  the  explicit  mle's  for  the  second  set  of  parameters,  and  finally 

describe  how  to  solve  for  the  mle's  of  Y  ,  0  0  ..  Section  4  provides 

s  so  si 

details  for  computing  these  solutions. 

For  S  -  1,...,J, 

p(z>0|s)  ■  irp(z>0|s,  u  »  0)  +  (1-x)p(z-0|s,  u  ■  1) 

w  (1-w)  (3.3) 

“  1+exptY„l  1+exptY  +ol  * 

8  3 
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Furthermore,  using  assumption  (1.3)  we  have 


p(r^»0|z«t,  s)  « 

p{r^-0|8. 


u>0)p(uB0|z-t,  s) 

+  p(r^>»0|s,  u»1  )p(u«1  |z-t,  s) 
w  1-w 


sz 


sz 


1+e 


st 


l+e 


St 


t 


(3.4) 


w  ■p(u-0Lz,s) 
sz 


n  explzY„] 

8 

1  +  exp[Y^] 


X  exp[zYg]  (1-x)  exp[z(Yg+a)] 

1  +  IexptY„]  *  1  +  exptY„  +  o] 

8  8 


It  +  (1-w) 


ZO.,  8, 

e  [H«  1 

Y  +o 
1  +  e  ® 


(3.5) 


Note  that  the  w__'s  define  the  conditional  distribution  of  the  unobserved 
sz 

covarlate  u  given  the  observed  treatment  assignment. 

Since  r^^^  is  observed  only  if  z  »  t,  the  likelihood  of  the  observed 
data  is 

N 

n  P<>^til*i  ”  ®'  ^s'  ^st^P^*i  "  Yg)p(s)  . 

i^l 

For  specified  (a,  6^,  w),  the  model  is  saturated,  so  the  mle's  of 

p(r  .  |z.  »  t,  s,  Y  ,  0  *.),  p(t,  ■  tjs,  Y  )  and  p(8)  are  the  corresponding 
observed  proportions*  the  mle  of  p(s  ■  j)  is  the  proportion  p(sBj)  of 
patients  in  stratum  j,  the  mle  of  p(z«o|s)  is  the  observed  proportion 
p(z  •  0|s)  of  patients  receiving  treatment  0  in  stratum  s,  and  for  t  *  0 


and  1,  the  mle  of  p(r^-0|z«t,8)  is  the  observed  proportion  ~  0|z  - 

t,  s)  of  patients  with  response  r^  >  0  among  patients  who  received 
treatment  z  =  t  in  stratum  s* 

For  each  stratum  s,  equation  (3.3)  with  p(z  -  o|s)  substituted  for 
p(z  >  0|s)  may  be  solved  for  the  mle  of  without  reference  to  any  other 

equation.  Having  found  Y  and  calculated  w_^,  equation  (3.4)  with 

S  Sb 

A  I  I  ^ 

p(r^  «  0|z  >  t,s)  substituted  for  p(r^  >  0|z  >  t,s)  may  be  solved  for  0^^ 
without  reference  to  any  other  equation.  Since  equations  (3.3)  and  (3.4)  are 
of  the  same  general  form,  a  single  one  dimensional  Newton-step  algorithm 
yields  maximum  lilcellhood  estimates  of  the  parameters  Y^, 
s  -  1 . J. 


The  mle  of  the  average  response  to  treatment  t  In  stratum  a  Is 
p(r^«0|s)^(r^*0|s,  u»0)ir  +  (1-w)p{r^-0|s,  u«1 ) 


1-x 


A 


a  ’ 

m  )+e  ^ 


Thus,  the  mle  of  the  average  response  to  treatmnt  t  is  given  by  the 

directly  adjusted  rate 

J 

^4.  ”  I  p(r  -0|8-j)p(s-j)  (3.6) 

j-1 

where  p(8»j)  is  the  proportion  of  patients  in  stratum  j.  Values  of  the 

A 

estimator  are  displayed  in  Table  2  for  our  data  and  various  assumptions 


about  the  sensitivity  parameters  (w,  a,  5^, 


4.  MAXIMUM  LIKEHHOOD  COMPUTATIONS 

Equations  (3.3)  and  (3.4)  are  both  of  the  form 

a  1-a 

P  *7-9  ^--5^  • 

l+e  1+e 


(4.1 ) 


To  find  9  for  fixed  a,  b  and  p,  we  use  the  following  Gauss-Newton 
iteration 


1-a 

a 

9  +b 

1+e  ” 

+  9  "  P 

.  n 

1+e 

9  +b 

9 

(1-a)e  " 

n 

1  *  ® 

9  +b  ^ 

9  ^ 

(l+e  ”  ) 

(1+e  ") 

(4.2) 


The  iteration  (4.2)  can  be  easily  programmed  on  a  pocket  calculator. 

A  A  A 

To  calculate  (Y  »  0  ®  )  fo*'  fixed  (n,  a,  0  ,  o  ): 

S  8U  81  1  Z 

A  I 

a)  Solve  (3.3)  for  Y  using  (4.2)  with  p  ■  p(zsO|s),  a  ■  ir,  b  «=■  o. 

s 

b)  For  t  =  0,1  find  w^^  using  (3.5). 

c)  For  t  =  0,1  solve  (3.4)  for  8  .  using  (4.2)  with  p  ■  p(r,.«0|z>=t,s), 
a  -  Wg^,  b  - 

Steps  a,  b,  and  c  are  repeated  for  S  »  1,...,J  and  the  results  combined 
using  (3.6). 

Although  the  iteration  (4.2)  can  easily  be  performed  using  a  hand 
calculator,  a  simple  closed  form  approximation  is  convenient  for  obtaining  a 
good  starting  value,  especially  if  many  sensitivity  analyses  are  to  be 
performed.  Specifically,  the  solution  9  of  equation  (4.1)  can  be  closely 
approximated  by  a  Taylor  approximation,  providing  .05  <  p  <  .95. 

Define  L  as  the  logit 

L  -  log 


Equation  (4«1)  may  be  rewritten 


-  a  1-a  1 

- 5+h - T  * 

1+e  1+e  1-fe“ 


(4.3) 


Implicit  differentiation  (e.g. ,  Hildebrand  {1962)«  page  339)  of  (4.3) 


yields 


AT  6  >  0.  p  equals 


and  therefore  L  equals 


(1-»a^) 

ae^  ^  (1»a)e^^ 
(I4e®)'  (1^®"^' 


a  1-a 


lo9.  —  -  l„  ,  ~y 


Expanding  around  Lq  yields  the  closed  form  approximation 


**0 

(14«  ) 

- b  • 

L  4.  (1-*)» 

*  h  2 

(l+e*^) 


(4.4) 
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